Note of Stochastic Method

Based on the book Handbook of Stochastic Methods: for Physics, Chemistry and the Natural Sciences.

Wiener-Khinchin therom

The autocorrelation function is defined as

And the power spectrum is defined as

We have:

The power spectrum and the autocorrelation function are the fourier transformation of each other.

Markov process

Definition and Chapman-Kolgomorov equation

Definition: $p(x_1,t_1;x_2,t_2,...|y_1,{\tau }_1;y_2,{\tau }_2,...)=p(x_1,t_1;x_2,t_2,...|y_1,{\tau }_1)$$p(x_1,t_1;x_2,t_2,...|y_1,\tau_1;y_2,\tau_2,...)=p(x_1,t_1;x_2,t_2,...|y_1,\tau_1)$.

Chapman-Kolgomorov eq. $p(x_1,t_1|x_3,t_3)=\int d{x}_{2}p(x_1,t_1|x_2,t_2)p(x_2,t_2|x_3,t_3)$$p(x_1,t_1|x_3,t_3)=\int d x_2~p(x_1,t_1|x_2,t_2)p(x_2,t_2|x_3,t_3)$.

Definition of continuous Markov process: $\underset{\mathrm{\Delta }t\to 0}{lim}\frac{1}{\mathrm{\Delta }t}{\int }_{|x-z|>ϵ}dxp(x,t+\mathrm{\Delta }t|z,t)=0$$\lim_{\Delta t\rightarrow 0}{1\over \Delta t}\int_{|x-z|>\epsilon} dx~p(x,t+\Delta t|z,t)=0$.

"Differential C-K eq":

where:

$W$$W$ term: jump term. Only keeping this term, the C-K eq reduces into "the master eq.". --- descrete $A$$A$ term: drift term. Only keeping this term, with a initial condition $p(\mathbf{z},t|\mathbf{y},t^{\prime })=\delta (\mathbf{z}-\mathbf{y})$$p(\bold z,t|\bold y,t')=\delta(\bold z-\bold y)$ can we get a deterministic trajectory. --- continuous $B$$B$ term: diffuse term. Fokker-Plank eq: without jump term: --- continuous

Some examples of Markov process

Wiener process: C-K eq:

with the initial condition $p(x_0,t_0)=\delta (x-x_0)$$p(x_0,t_0)=\delta(x-x_0)$ we have:

So we have:

term with square root exists, so $X$$X$ is non-differentiable.

Ornstein-Uhlenbeck process: C-K eq:

The Ito calculus and stochastic differential equation

The aim

The aim is to solve the DE: $\frac{\mathrm{d}}{\mathrm{d}t}x=a(x,t)+b(x,t)\xi (t)$${\dd\over \dd t}x=a(x,t)+b(x,t)\xi(t)$, where $\xi (t)$$\xi(t)$ satisfy:

To do so, we need the integral of stochastic term: $W=\int \xi (t)\mathrm{d}t$$W=\int \xi(t)\dd t$. It is obvious that $W$$W$ is a continuous Markov process. It should be characterized by Fokker-Planck eq. Considering the fact:

we have $A=0$$A=0$ and $B=1$$B=1$. So $W$$W$ is a Wiener process. Wiener process is non-differentiable, so the origin DE is not valid. Thus we would write it in the following form:

Ito integral and Ito SDE

Let the integral of $b(x,t)\mathrm{d}W$$b(x,t)\dd W$ to be:

note:

1. here we choose $b$$b$ to be at the left end of the interval, thus $b$$b$ is more probable to be a nonanticipationg function (functions that $b_i$$b_i$ is independent of $W_{i+1}-W_i$$W_{i+1}-W_i$). And it seems strange but is true that the position of $b$$b$ affect the result of integral. (Difference between Ito form and Stratonovich form.)

   Choose $b(x,t)$$b(x,t)$ to be $W(t)$$W(t)$, then $S=\int W\mathrm{d}W$$S=\int W\dd W$$=lim\sum W({\tau }_i)[W(t_i)-W(t_{i-1})]$$=\lim\sum W(\tau_i)[W(t_i)-W(t_{i-1})]$. We have $⟨S⟩=lim\sum ({\tau }_i-t_i)$$\braket S=\lim\sum(\tau_i-t_i)$ depending on ${\tau }_i$$\tau_i$.

2. $\mathrm{m}\mathrm{s}\underset{n\to \mathrm{\infty }}{lim}$$\msl$ is subtle. On one hand, it discard some stochastic term that constrate in one value with probability 1; on the other hand, it still gives back a stochastic term that is indetermined.

It can be proved that, for indetermined functions $a,b$$a,b$,

Correspondingly, one can define Ito stochastic differential equation:

whose solution is Ito integral.

Stratonovich integral

One can use another choice of ${\tau }_i$$\tau_i$ to be ${\tau }_i=\frac{1}{2}(t_i+t_{i+1})$$\tau_i={1\over 2}(t_i+t_{i+1})$. Then Stratonovich integral was formed (only $x$$x$ changes, when the functions are singular, $t$$t$'s choice may matter. ):

Attention, now,$b$$b$ is not a indetermined function, thus all the good lemmas for Ito integral fail.

Stratonovich SDE:

whose solution is Stratonovich integral.

The relationship between Ito integral and Strtonovich integral can be derived as follow:

Here we ignore the quantities with order higher than $\mathrm{\Delta }{W}^2$$\Delta W^2$. Now all the functions are indetermined, thus $\mathrm{\Delta }{W}^2=\mathrm{\Delta }t$$\Delta W^2=\Delta t$. Therefore,

The relationship between SDE and FPE

Under Ito explanation,

corresponds to:

Multi-variable case:

Adiabatic elimination of fast variables

Kramers equation

The EOM of a particle affected by stochastic force in potential:

Given that $m$$m$ is a small quantity, $v$$v$ will be thermalized in a short time, what we need to do is to systematically eliminate it to get an equation of $x$$x$. Or, we need to find the FPE of $\tilde{p}(x,t)=\int \mathrm{d}vp(x,v,t)$$\tilde p(x,t)=\int \dd v \ p(x,v,t)$.

The corresponding FPE is:

Scale the variables: $y=x\sqrt{m/kT},u=v\sqrt{m/kT},U(y)=U(x)/kT$$y=x\sqrt{m/kT},u=v\sqrt{m/kT},U(y)=U(x)/kT$, and introduce $\gamma =\beta /m$$\gamma=\beta/m$, a large quantity.

expand $p$$p$ into series: $p=p_0+p_1{\gamma }^{-1}+p_2{\gamma }^{-2}+...$$p=p_0+p_1\gamma^{-1}+p_2\gamma^{-2}+...$.

Order 0:

we employ the solution (there seems to be another solution, but somehow it is ignored):

Plug it into the equation of order 1:

Integrate the both side over $u$$u$ to eliminate the odd terms and boundary term, one gets:

Thus order 1:

the solution is (again something is left):

Order 2:

Similarly, integrate it:

Now plug $\tilde{p}=\int \mathrm{d}u[p_0+{\gamma }^{-1}{p}_1+O({\gamma }^{-2})]$$\tilde p=\int\dd u[ p_0+\gamma^{-1}p_1+O(\gamma^{-2})]$$=\sqrt{2\pi }[\varphi (y)+{\gamma }^{-1}\psi (y,t)+O({\gamma }^{-2})]$$=\sqrt{2\pi}[\phi(y)+\gamma^{-1}\psi(y,t)+O(\gamma^{-2})]$ into this condition:

This is "Kramers equation".

Stratonovich integral is the limit of nonwhite noise process

One stochastic differential equation reads:

where ${\alpha }_0(t)=\gamma \alpha ({\gamma }^{2}t)$$\alpha_0(t)=\gamma\alpha(\gamma^2t)$, $\alpha (t)$$\alpha(t)$ is a stationary stochastic process with

Aim and conclusion: under the limit $\gamma \to \mathrm{\infty }$$\gamma\to \infty$, the equation before reduces to Stratonovich stochastic differential equation:

This shows that stratonovich integral is the limit of nonwhite noise process. In this case, $\alpha$$\alpha$ can be regarded as the fast variable.

Proof

One assume that $\alpha (t)$$\alpha(t)$ obeys:

$${\partial }_{t}p(\alpha (t))={\hat{L}}_{1}p(\alpha (t))=[-{\partial }_{\alpha }A(\alpha )+\frac{1}{2}{\partial }_{\alpha }^{2}B(\alpha )]p(\alpha (t))$$

i.e. $\alpha (t)$$\alpha(t)$ is a diffusion process. Denote $\alpha ({\gamma }^{2}t)$$\alpha(\gamma^2 t)$ by $\alpha$$\alpha$, one can get FPE of the pair $(x,\alpha )$$(x,\alpha)$:

$$\begin{array}{c}{\partial }_{t}p(x,\alpha ,t)=({\gamma }^2{\hat{L}}_1+\gamma {\hat{L}}_2+{\hat{L}}_3)p(x,\alpha ,t)\\ \mathrm{w}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e}{\hat{L}}_2=-\alpha {\partial }_{x}b(x){\hat{L}}_3=-{\partial }_{x}a(x)\end{array}$$

even if $\alpha$$\a$ is not a diffusion process, one is able to derive it.

The first order approximation: ${\gamma }^2{\hat{L}}_{1}p=0$$\gamma^2\hat L_1 p=0$. The solution should be $p(x,\alpha ,t)=p_s(\alpha )\varphi (x,t)$$p(x,\a,t)=p_s(\a)\phi(x,t)$, where $p_s(\alpha )$$p_s(\a)$ satisfies ${\hat{L}}_1{p}_s(\alpha )=0$$\hat L_1 p_s(\a)=0$. So we use the operator $\hat{P}p(x,\alpha ,t)=p_s(\alpha )\int \mathrm{d}\alpha p(x,\alpha ,t)$$\hat Pp(x,\a,t)=p_s(\a)\int \dd \a \ p(x,\a,t)$ to characterize the approximation: $v=\hat{P}p,w=(1-\hat{P})p,p=v+w$$v=\hat P p, w=(1-\hat P)p,p=v+w$. $v=p_s(\alpha )\tilde{p}(x,t)$$v=p_s(\a)\tilde p(x,t)$, what we need is FPE of $\tilde{p}$$\tilde p$, thus we need to cancel $w$$w$.

Apply $\hat{P}$$\hat P$ and $1-\hat{P}$$1-\hat P$ in both side of:

$${\partial }_t(v+w)=({\gamma }^2{\hat{L}}_1+\gamma {\hat{L}}_2+{\hat{L}}_3)(v+w)$$

and use lemmas:

$$\begin{array}{c}\hat{P}{\hat{L}}_1={\hat{L}}_1\hat{P}=0\\ \hat{P}{\hat{L}}_2\hat{P}=0\\ \hat{P}{\hat{L}}_3={\hat{L}}_3\hat{P}\end{array}$$

one can get

$$\begin{array}{c}{\partial }_{t}v={\hat{L}}_{3}v+\gamma \hat{P}{\hat{L}}_{2}w\\ {\partial }_{t}w=\gamma {\hat{L}}_{2}v+{\gamma }^2{\hat{L}}_{1}w+\gamma (1-\hat{P}){\hat{L}}_{2}w+{\hat{L}}_{3}w\end{array}$$

Proof of lemmas

Obviously ${\hat{L}}_1\hat{P}=0$$\hat L_1\hat P=0$ and ${\hat{L}}_3\hat{P}=\hat{P}{\hat{L}}_3$$\hat L_3\hat P=\hat P\hat L_3$. Regarding $\hat{P}{\hat{L}}_1=0$$\hat P\hat L_1=0$: If ${\hat{L}}_3$$\hat L_3$ has a complete set of eigenfunctions and corresponding eigenvalues are all real, $\hat{P}=\underset{t\to \mathrm{\infty }}{lim}\exp (-t{\hat{L}}_1)$$\hat P=\lim_{t\to\infty}\exp(-t\hat L_1)$. Then the exchangeablity is obvious. For other cases idk.

Regarding $\hat{P}{\hat{L}}_2\hat{P}=0$$\hat P\hat L_2\hat P=0$: one can easily get:

$$\hat{P}{\hat{L}}_2\hat{P}f(x,\alpha ,t)=[\mathrm{s}\mathrm{t}\mathrm{h}\mathrm{o}\mathrm{f}x,t]\int \mathrm{d}\alpha \alpha p_s(\alpha )$$

From $⟨\alpha ⟩=0$$\braket \a=0$, one can know the above formula equals to 0;

Laplacian transformation of the two equation and eliminate $w$$w$ (let $\mathcal{L}w(0)=0$$\mathcal L{w}(0)=0$):

$$sv={\hat{L}}_{3}v-\gamma \hat{P}{\hat{L}}_2[-s+{\gamma }^2{\hat{L}}_1+\gamma (1-\hat{P}){\hat{L}}_2+{\hat{L}}_3]\gamma {\hat{L}}_{2}v+v(0)$$

here $v\leftarrow \mathcal{L}\{v(t)\}(s)$$v\gets \mathcal L \{v(t)\}(s)$. Keep the main terms, one gets:

$$\begin{array}{c}sv=({\hat{L}}_3-\hat{P}{\hat{L}}_2{\hat{L}}_1^{-1}{\hat{L}}_2)v+v(0)\\ \mathrm{i}.\mathrm{e}.{\partial }_{t}v=({\hat{L}}_3-\hat{P}{\hat{L}}_2{\hat{L}}_1^{-1}{\hat{L}}_2)v\end{array}$$

 

General Method

tbd...

Approximation between master equation and Fokker-Planck equation

Master eq as an approximation of FPE

One can always use master equation to approximate Fokker-Planck equation. Considering the following jumping term:

$\mathrm{\Phi }(y;x,t)\to 0$$\Phi(y;x,t)\to 0$ when $y\to \pm \mathrm{\infty }$$y\to \pm \infty$ and $\int \mathrm{\Phi }(y;x,t)\mathrm{d}y=1$$\int\Phi(y;x,t)\dd y=1$ and $\int y\mathrm{\Phi }(y;x,t)\mathrm{d}y=0$$\int y\Phi(y;x,t)\dd y=0$. We have:

As $\delta \to 0$$\delta\to 0$, one have:

Practically, one can use the birth-death equation to approximate FPE.

$\delta$$\delta$'s with footnote are Dirac delta function.

FPE as an approximation of master eq

Kramers-Moyal expansion is easy to carry on but not rigorous. For a master eq:

One can use the following FPE as a substitution:

Proof (not rigorous at all):

Let $\mathrm{\Delta }x=x^{\prime }-x$$\Delta x=x'-x$ and $W(x|x^{\prime },t)=W(\mathrm{\Delta }x;x^{\prime },t)$$W(x|x',t)=W(\Delta x;x',t)$. One has RHS:

$$\begin{array}{c}\int \mathrm{d}\mathrm{\Delta }x[W(\mathrm{\Delta }x;x+\mathrm{\Delta }x,t)p(x+\mathrm{\Delta }x,t)-W(-\mathrm{\Delta }x;x,t)p(x,t)]\\ =\int \mathrm{d}\mathrm{\Delta }x[W(\mathrm{\Delta }x;x+\mathrm{\Delta }x,t)p(x+\mathrm{\Delta }x,t)-W(\mathrm{\Delta }x;x,t)p(x,t)]\\ =\int \mathrm{d}\mathrm{\Delta }x\sum _{n=1}^{\mathrm{\infty }}\frac{1}{n!}(\mathrm{\Delta }x{\partial }_x{)}^{n}W(\mathrm{\Delta }x;x,t)p(x,t)\\ =\sum _{n=1}^{\mathrm{\infty }}{\partial }_x^n[\frac{1}{n!}\int \mathrm{d}\mathrm{\Delta }x\mathrm{\Delta }{x}^{n}W(\mathrm{\Delta }x;x,t)]p(x,t)\end{array}$$

In the second line the identity ${\int }_{\mathbb{R}}\mathrm{d}xW(x)={\int }_{\mathbb{R}}\mathrm{d}xW(-x)$$\int_\R\dd xW(x)=\int_\R\dd x W(-x)$ is used. Abandon terms with order higher than 2, one get

$${\partial }_{t}p={\partial }_{x}Ap+\frac{1}{2}{\partial }_x^{2}Bp$$

where:

$$\begin{array}{c}A=\int \mathrm{d}\mathrm{\Delta }x\mathrm{\Delta }xW(\mathrm{\Delta }x;x,t)=-\int \mathrm{d}{x}^{\prime }(x^{\prime }-x)W(x^{\prime }|x,t)\\ B=\int \mathrm{d}\mathrm{\Delta }x\mathrm{\Delta }{x}^{2}W(\mathrm{\Delta }x;x,t)=\int \mathrm{d}{x}^{\prime }(x^{\prime }-x{)}^{2}W(x^{\prime }|x,t)\end{array}$$

This process is not regorous-- one is not even clear the asymptotic process of this approximation. But obviously it is valid when $W(\mathrm{\Delta }x;x,t)$$W(\Delta x;x,t)$ as a function of $\mathrm{\Delta }x$$\Delta x$ is continuous and limited in a small range-- the high-ordered moment vanishes.

Van Kampen's system size expansion is valid. "The essential point is that the size of the jump is expressed in terms of the extensive quantity $\mathrm{\Delta }a$$\Delta a$, but the dependence on $a^{\prime }$$a'$ is better expressed in terms of the intenshive variable $x$$x$." Van Kampen expect that the jumping term obeys:

Where $\mathrm{\Omega }$$\Omega$ is the size of the system (such as volume or area). ($W(\mathrm{\Delta }x;x,t)$$W(\Delta x;x,t)$ is redefined.) Taking chemical reaction as an instance, this seems reasonable.

The conclusion is: one can replace the master eq with:

brief proof:

Plug $x=...\varphi +...z$$x=...\phi+...z$ into the series in Kramers-Moyal expansion. Then expand $W$$W$ again into series of ${\mathrm{\Omega }}^{-1/2}$$\Omega^{-1/2}$. The condition of $\varphi (t)$$\phi(t)$ cancel ${\mathrm{\Omega }}^{1/2}$$\Omega^{1/2}$ term in both sides. Only let zero-ordered terms servive and one get the final expr.

It can be proved that in the lowest order, when $W$$W$ meet the requirement of van Kampen's expansion, the result of Kramers-Moyal expansion and that of van Kampen's are consistent.

Critical fluctuation and critical slowing down: tbd...

Multi-variable birth-death master equation

The most apparent example of the birth-death process with multiple variables is multiple chemical reactions:

where $A$$A$ denotes the index of the reaction, and $\mathcal{X}$$\mathcal X$ denotes the species of substance. One use $\overrightarrow{x}$$\vec x$ to denote the amount of every substance and ${\overrightarrow{r}}^A$$\vec r^A$ to denote ${\overrightarrow{M}}^A-{\overrightarrow{N}}^A$$\vec M^A-\vec N^A$. Thus

To simplify the problem, one set $k$$k$ to be independent of $t$$t$ and try to find the stationary solutions. The general stationary solution of the master equation can be found by some method, and the result is called Kirchoff's solution.

Stationary solution with detailed balance

Detailed balance requires the the following conditions:

$p(\overrightarrow{x}+{\overrightarrow{r}}^A+{\overrightarrow{r}}^B)=p(\overrightarrow{x}+{\overrightarrow{r}}^B+{\overrightarrow{r}}^A)$$p(\vec x+\vec r^A+\vec r^B)=p(\vec x+\vec r^B+\vec r^A)$ is automatically satisfied. With the following condition, detailed balance can be ensured:

• two different (in the commutative aspect) path $A_1,A_2,...$$A_1,A_2,...$ and $B_1,B_2,...$$B_1,B_2,...$ such that $\sum _i{\overrightarrow{r}}^{A_i}=\sum _i{\overrightarrow{r}}^{B_i}$$\sum_i\vec r^{A_i}=\sum_i\vec r^{B_i}$, if exist, satisfy $\prod _i\frac{k_{A_i}^{+}}{k_{A_i}^{-}}=\prod _i\frac{k_{B_i}^{+}}{k_{B_i}^{-}}$$\prod_i{k^+_{A_i}\over k^-_{A_i}}=\prod_i{k^+_{B_i}\over k^-_{B_i}}$. This condition ensure $p(\overrightarrow{x}+\sum _i{\overrightarrow{r}}^{A_i})=p(\overrightarrow{x}+\sum _i{\overrightarrow{r}}^{B_i})$$p(\vec x+\sum_i\vec r^{A_i})=p(\vec x+\sum_i\vec r^{B_i})$.

The general stationary solution can be written as a poissonian distribution times a function $f(\overrightarrow{x})$$f(\vec x)$.

where ${\lambda }_a$$\lambda_a$ is the solution of the following linear equations:

And $f(\overrightarrow{x})$$f(\vec x)$ satisfied $f(\overrightarrow{x})=f(\overrightarrow{x}+{\overrightarrow{r}}^A),\mathrm{\forall }A$$f(\vec x)=f(\vec x+\vec r^A),~\forall A$. $f$$f$ distinguish out a subspace, for instance, in the reaction $A\iff B$$A\Lrarr B$, $f(a,b)=\delta (a+b-N)$$f(a,b)=\delta(a+b-N)$ ensure the conservation.

FPE approximation of multi-variable master eq

Kramers-Moyal expansion: